Content independent data compression method and system

ABSTRACT

Systems and methods for providing fast and efficient data compression using a combination of content independent data compression and content dependent data compression. In one aspect, a method for compressing data comprises the steps of: analyzing a data block of an input data stream to identify a data type of the data block, the input data stream comprising a plurality of disparate data types; performing content dependent data compression on the data block, if the data type of the data block is identified; performing content independent data compression on the data block, if the data type of the data block is not identified.

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a Continuation-In-Part of U.S. patentapplication Ser. No. 09/705,446, filed on Nov. 3, 2000, which is aContinuation of U.S. patent application Ser. No. 09/210,491, filed onDec. 11, 1998, which is now U.S. Pat. No. 6,195,024, issued on Feb. 27,2001.

BACKGROUND

[0002] 1. Technical Field

[0003] The present invention relates generally to a data compression anddecompression and, more particularly, to systems and methods for datacompression using content independent and content dependent datacompression and decompression.

[0004] 2. Description of Related Art

[0005] Information may be represented in a variety of manners. Discreteinformation such as text and numbers are easily represented in digitaldata. This type of data representation is known as symbolic digitaldata. Symbolic digital data is thus an absolute representation of datasuch as a letter, figure, character, mark, machine code, or drawing.

[0006] Continuous information such as speech, music, audio, images andvideo, frequently exists in the natural world as analog information. Asis well known to those skilled in the art, recent advances in very largescale integration (VLSI) digital computer technology have enabled bothdiscrete and analog information to be represented with digital data.Continuous information represented as digital data is often referred toas diffuse data. Diffuse digital data is thus a representation of datathat is of low information density and is typically not easilyrecognizable to humans in its native form.

[0007] There are many advantages associated with digital datarepresentation. For instance, digital data is more readily processed,stored, and transmitted due to its inherently high noise immunity. Inaddition, the inclusion of redundancy in digital data representationenables error detection and/or correction. Error detection and/orcorrection capabilities are dependent upon the amount and type of dataredundancy, available error detection and correction processing, andextent of data corruption.

[0008] One outcome of digital data representation is the continuing needfor increased capacity in data processing, storage, and transmittal.This is especially true for diffuse data where increases in fidelity andresolution create exponentially greater quantities of data. Datacompression is widely used to reduce the amount of data required toprocess, transmit, or store a given quantity of information. In general,there are two types of data compression techniques that may be utilizedeither separately or jointly to encode/decode data: lossless and lossydata compression.

[0009] Lossy data compression techniques provide for an inexactrepresentation of the original uncompressed data such that the decoded(or reconstructed) data differs from the original unencoded/uncompresseddata. Lossy data compression is also known as irreversible or noisycompression. Entropy is defined as the quantity of information in agiven set of data. Thus, one obvious advantage of lossy data compressionis that the compression ratios can be larger than the entropy limit, allat the expense of information content. Many lossy data compressiontechniques seek to exploit various traits within the human senses toeliminate otherwise imperceptible data. For example, lossy datacompression of visual imagery might seek to delete information contentin excess of the display resolution or contrast ratio.

[0010] On the other hand, lossless data compression techniques providean exact representation of the original uncompressed data. Simplystated, the decoded (or reconstructed) data is identical to the originalunencoded/uncompressed data. Lossless data compression is also known asreversible or noiseless compression. Thus, lossless data compressionhas, as its current limit, a minimum representation defined by theentropy of a given data set.

[0011] There are various problems associated with the use of losslesscompression techniques. One fundamental problem encountered with mostlossless data compression techniques are their content sensitivebehavior. This is often referred to as data dependency. Data dependencyimplies that the compression ratio achieved is highly contingent uponthe content of the data being compressed. For example, database filesoften have large unused fields and high data redundancies, offering theopportunity to losslessly compress data at ratios of 5 to 1 or more. Incontrast, concise software programs have little to no data redundancyand, typically, will not losslessly compress better than 2 to 1.

[0012] Another problem with lossless compression is that there aresignificant variations in the compression ratio obtained when using asingle lossless data compression technique for data streams havingdifferent data content and data size. This process is known as naturalvariation.

[0013] A further problem is that negative compression may occur whencertain data compression techniques act upon many types of highlycompressed data. Highly compressed data appears random and many datacompression techniques will substantially expand, not compress this typeof data.

[0014] For a given application, there are many factors that govern theapplicability of various data compression techniques. These factorsinclude compression ratio, encoding and decoding processingrequirements, encoding and decoding time delays, compatibility withexisting standards, and implementation complexity and cost, along withthe adaptability and robustness to variations in input data. A directrelationship exists in the current art between compression ratio and theamount and complexity of processing required. One of the limitingfactors in most existing prior art lossless data compression techniquesis the rate at which the encoding and decoding processes are performed.Hardware and software implementation tradeoffs are often dictated byencoder and decoder complexity along with cost.

[0015] Another problem associated with lossless compression methods isdetermining the optimal compression technique for a given set of inputdata and intended application. To combat this problem, there are manyconventional content dependent techniques that may be utilized. Forinstance, file type descriptors are typically appended to file names todescribe the application programs that normally act upon the datacontained within the file. In this manner data types, data structures,and formats within a given file may be ascertained. Fundamentallimitations with this content dependent technique include:

[0016] (1) the extremely large number of application programs, some ofwhich do not possess published or documented file formats, datastructures, or data type descriptors;

[0017] (2) the ability for any data compression supplier or consortiumto acquire, store, and access the vast amounts of data required toidentify known file descriptors and associated data types, datastructures, and formats; and

[0018] (3) the rate at which new application programs are developed andthe need to update file format data descriptions accordingly.

[0019] An alternative technique that approaches the problem of selectingan appropriate lossless data compression technique is disclosed, forexample, in U.S. Pat. No. 5,467,087 to Chu entitled “High Speed LosslessData Compression System” (“Chu”). FIG. 1 illustrates an embodiment ofthis data compression and decompression technique. Data compression 1comprises two phases, a data pre-compression phase 2 and a datacompression phase 3. Data decompression 4 of a compressed input datastream is also comprised of two phases, a data type retrieval phase 5and a data decompression phase 6. During the data compression process 1,the data pre-compressor 2 accepts an uncompressed data stream,identifies the data type of the input stream, and generates a data typeidentification signal. The data compressor 3 selects a data compressionmethod from a preselected set of methods to compress the input datastream, with the intention of producing the best available compressionratio for that particular data type.

[0020] There are several limitations associated with the Chu method. Onesuch limitation is the need to unambiguously identify various datatypes. While these might include such common data types as ASCII,binary, or unicode, there, in fact, exists a broad universe of datatypes that fall outside the three most common data types.

[0021] Examples of these alternate data types include: signed andunsigned integers of various lengths, differing types and precision offloating point numbers, pointers, other forms of character text, and amultitude of user defined data types. Additionally, data types may beinterspersed or partially compressed, making data type recognitiondifficult and/or impractical. Another limitation is that given a knowndata type, or mix of data types within a specific set or subset of inputdata, it may be difficult and/or impractical to predict which dataencoding technique yields the highest compression ratio.

[0022] Accordingly, there is a need for a data compression system andmethod that would address limitations in conventional data compressiontechniques as described above.

SUMMARY OF THE INVENTION

[0023] The present invention is directed to systems and methods forproviding fast and efficient data compression using a combination ofcontent independent data compression and content dependent datacompression. In one aspect of the invention, a method for compressingdata comprises the steps of:

[0024] analyzing a data block of an input data stream to identify a datatype of the data block, the input data stream comprising a plurality ofdisparate data types;

[0025] performing content dependent data compression on the data block,if the data type of the data block is identified;

[0026] performing content independent data compression on the datablock, if the data type of the data block is not identified.

[0027] In another aspect, the step of performing content independentdata compression comprises: encoding the data block with a plurality ofencoders to provide a plurality of encoded data blocks; determining acompression ratio obtained for each of the encoders; comparing each ofthe determined compression ratios with a first compression threshold;selecting for output the input data block and appending a nullcompression descriptor to the input data block, if all of the encodercompression ratios do not meet the first compression threshold; andselecting for output the encoded data block having the highestcompression ratio and appending a corresponding compression typedescriptor to the selected encoded data block, if at least one of thecompression ratios meet the first compression threshold.

[0028] In another aspect, the step of performing content dependentcompression comprises the steps of: selecting one or more encodersassociated with the identified data type and encoding the data blockwith the selected encoders to provide a plurality of encoded datablocks; determining a compression ratio obtained for each of theselected encoders; comparing each of the determined compression ratioswith a second compression threshold; selecting for output the input datablock and appending a null compression descriptor to the input datablock, if all of the encoder compression do not meet the secondcompression threshold; and selecting for output the encoded data blockhaving the highest compression ratio and appending a correspondingcompression type descriptor to the selected encoded data block, if atleast one of the compression ratios meet the second compressionthreshold.

[0029] In yet another aspect, the step of performing content independentdata compression on the data block, if the data type of the data blockis not identified, comprises the steps of: estimating a desirability ofusing of one or more encoder types based one characteristics of the datablock; and compressing the data block using one or more desirableencoders.

[0030] In another aspect, the step of performing content dependent datacompression on the data block, if the data type of the data block isidentified, comprises the steps of: estimating a desirability of usingof one or more encoder types based on characteristics of the data block;and compressing the data block using one or more desirable encoders.

[0031] In another aspect, the step of analyzing the data block comprisesanalyzing the data block to recognize one of a data type, datastructure, data block format, file substructure, and/or file types. Afurther step comprises maintaining an association between encoder typesand data types, data structures, data block formats, file substructure,and/or file types.

[0032] In yet another aspect of the invention, a method for compressingdata comprises the steps of:

[0033] analyzing a data block of an input data stream to identify a datatype of the data block, the input data stream comprising a plurality ofdisparate data types;

[0034] performing content dependent data compression on the data block,if the data type of the data block is identified;

[0035] determining a compression ratio of the compressed data blockobtained using the content dependent compression and comparing thecompression ratio with a first compression threshold; and

[0036] performing content independent data compression on the datablock, if the data type of the data block is not identified or if thecompression ratio of the compressed data block obtained using thecontent dependent compression does not meet the first compressionthreshold.

[0037] Advantageously, the present invention employs a plurality ofencoders applying a plurality of compression techniques on an input datastream so as to achieve maximum compression in accordance with thereal-time or pseudo real-time data rate constraint.

[0038] Thus, the output bit rate is not fixed and the amount, if any, ofpermissible data quality degradation is user or data specified.

[0039] These and other aspects, features and advantages of the presentinvention will become apparent from the following detailed descriptionof preferred embodiments, which is to be read in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0040]FIG. 1 is a block/flow diagram of a content dependent high-speedlossless data compression and decompression system/method according tothe prior art;

[0041]FIG. 2 is a block diagram of a content independent datacompression system according to one embodiment of the present invention;

[0042]FIGS. 3a and 3 b comprise a flow diagram of a data compressionmethod according to one aspect of the present invention, whichillustrates the operation of the data compression system of FIG. 2;

[0043]FIG. 4 is a block diagram of a content independent datacompression system according to another embodiment of the presentinvention having an enhanced metric for selecting an optimal encodingtechnique;

[0044]FIGS. 5a and 5 b comprise a flow diagram of a data compressionmethod according to another aspect of the present invention, whichillustrates the operation of the data compression system of FIG. 4;

[0045]FIG. 6 is a block diagram of a content independent datacompression system according to another embodiment of the presentinvention having an a priori specified timer that provides real-time orpseudo real-time of output data;

[0046]FIGS. 7a and 7 b comprise a flow diagram of a data compressionmethod according to another aspect of the present invention, whichillustrates the operation of the data compression system of FIG. 6;

[0047]FIG. 8 is a block diagram of a content independent datacompression system according to another embodiment having an a priorispecified timer that provides real-time or pseudo real-time of outputdata and an enhanced metric for selecting an optimal encoding technique;

[0048]FIG. 9 is a block diagram of a content independent datacompression system according to another embodiment of the presentinvention having an encoding architecture comprising a plurality of setsof serially cascaded encoders;

[0049]FIGS. 10a and 10 b comprise a flow diagram of a data compressionmethod according to another aspect of the present invention, whichillustrates the operation of the data compression system of FIG. 9;

[0050]FIG. 11 is block diagram of a content independent datadecompression system according to one embodiment of the presentinvention;

[0051]FIG. 12 is a flow diagram of a data decompression method accordingto one aspect of the present invention, which illustrates the operationof the data compression system of FIG. 11;

[0052]FIGS. 13a and 13 b comprise a block diagram of a data compressionsystem comprising content dependent and content independent datacompression, according to an embodiment of the present invention;

[0053]FIGS. 14a-14 d comprise a flow diagram of a data compressionmethod using both content dependent and content independent datacompression, according to one aspect of the present invention;

[0054]FIGS. 15a and 15 b comprise a block diagram of a data compressionsystem comprising content dependent and content independent datacompression, according to another embodiment of the present invention;

[0055]FIGS. 16a-16 d comprise a flow diagram of a data compressionmethod using both content dependent and content independent datacompression, according to another aspect of the present invention;

[0056]FIGS. 17a and 17 b comprise a block diagram of a data compressionsystem comprising content dependent and content independent datacompression, according to another embodiment of the present invention;and

[0057]FIGS. 18a-18 d comprise a flow diagram of a data compressionmethod using both content dependent and content independent datacompression, according to another aspect of the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0058] The present invention is directed to systems and methods forproviding data compression and decompression using content independentand content dependent data compression and decompression. In thefollowing description, it is to be understood that system elementshaving equivalent or similar functionality are designated with the samereference numerals in the Figures. It is to be further understood thatthe present invention may be implemented in various forms of hardware,software, firmware, or a combination thereof. In particular, the systemmodules described herein are preferably implemented in software as anapplication program that is executable by, e.g., a general purposecomputer or any machine or device having any suitable and preferredmicroprocessor architecture. Preferably, the present invention isimplemented on a computer platform including hardware such as one ormore central processing units (CPU), a random access memory (RAM), andinput/output (I/O) interface(s). The computer platform also includes anoperating system and microinstruction code. The various processes andfunctions described herein may be either part of the microinstructioncode or application programs which are executed via the operatingsystem. In addition, various other peripheral devices may be connectedto the computer platform such as an additional data storage device and aprinting device.

[0059] It is to be further understood that, because some of theconstituent system components described herein are preferablyimplemented as software modules, the actual system connections shown inthe Figures may differ depending upon the manner in which the systemsare programmed. It is to be appreciated that special purposemicroprocessors may be employed to implement the present invention.Given the teachings herein, one of ordinary skill in the related artwill be able to contemplate these and similar implementations orconfigurations of the present invention.

[0060] Referring now to FIG. 2 a block diagram illustrates a contentindependent data compression system according to one embodiment of thepresent invention. The data compression system includes a counter module10 that receives as input an uncompressed or compressed data stream. Itis to be understood that the system processes the input data stream indata blocks that may range in size from individual bits through completefiles or collections of multiple files. Additionally, the data blocksize may be fixed or variable. The counter module 10 counts the size ofeach input data block (i.e., the data block size is counted in bits,bytes, words, any convenient data multiple or metric, or any combinationthereof).

[0061] An input data buffer 20, operatively connected to the countermodule 10, may be provided for buffering the input data stream in orderto output an uncompressed data stream in the event that, as discussed infurther detail below, every encoder fails to achieve a level ofcompression that exceeds an a priori specified minimum compression ratiothreshold. It is to be understood that the input data buffer 20 is notrequired for implementing the present invention.

[0062] An encoder module 30 is operatively connected to the buffer 20and comprises a set of encoders E1, E2, E3 . . . En. The encoder set E1,E2, E3 . . . En may include any number “n” of those lossless encodingtechniques currently well known within the art such as run length,Huffman, Lempel-Ziv Dictionary Compression, arithmetic coding, datacompaction, and data null suppression. It is to be understood that theencoding techniques are selected based upon their ability to effectivelyencode different types of input data. It is to be appreciated that afull complement of encoders are preferably selected to provide a broadcoverage of existing and future data types.

[0063] The encoder module 30 successively receives as input each of thebuffered input data blocks (or unbuffered input data blocks from thecounter module 10). Data compression is performed by the encoder module30 wherein each of the encoders E1 . . . En processes a given input datablock and outputs a corresponding set of encoded data blocks. It is tobe appreciated that the system affords a user the option toenable/disable any one or more of the encoders E1 . . . En prior tooperation. As is understood by those skilled in the art, such featureallows the user to tailor the operation of the data compression systemfor specific applications. It is to be further appreciated that theencoding process may be performed either in parallel or sequentially. Inparticular, the encoders E1 through En of encoder module 30 may operatein parallel (i.e., simultaneously processing a given input data block byutilizing task multiplexing on a single central processor, via dedicatedhardware, by executing on a plurality of processor or dedicated hardwaresystems, or any combination thereof). In addition, encoders E1 throughEn may operate sequentially on a given unbuffered or buffered input datablock. This process is intended to eliminate the complexity andadditional processing overhead associated with multiplexing concurrentencoding techniques on a single central processor and/or dedicatedhardware, set of central processors and/or dedicated hardware, or anyachievable combination. It is to be further appreciated that encoders ofthe identical type may be applied in parallel to enhance encoding speed.For instance, encoder E1 may comprise two parallel Huffman encoders forparallel processing of an input data block.

[0064] A buffer/counter module 40 is operatively connected to theencoding module 30 for buffering and counting the size of each of theencoded data blocks output from 30 encoder module 30. Specifically, thebuffer/counter 30 comprises a plurality of buffer/counters BC1, BC2, BC3. . . BCn, each operatively associated with a corresponding one of theencoders E1 . . . En. A compression ratio module 50, operativelyconnected to the output buffer/counter 40, determines the compressionratio obtained for each of the enabled encoders E1 . . . En by takingthe ratio of the size of the input data block to the size of the outputdata block stored in the corresponding buffer/counters BC1 . . . BCn. Inaddition, the compression ratio module 50 compares each compressionratio with an a priori-specified compression ratio threshold limit todetermine if at least one of the encoded data blocks output from theenabled encoders E1 . . . En achieves a compression that exceeds an apriori-specified threshold. As is understood by those skilled in theart, the threshold limit may be specified as any value inclusive of dataexpansion, no data compression or expansion, or any arbitrarily desiredcompression limit. A description module 60, operatively coupled to thecompression ratio module 50, appends a corresponding compression typedescriptor to each encoded data block which is selected for output so asto indicate the type of compression format of the encoded data block.

[0065] The operation of the data compression system of FIG. 2 will nowbe discussed in further detail with reference to the flow diagram ofFIGS. 3a and 3 b. A data stream comprising one or more data blocks isinput into the data compression system and the first data block in thestream is received (step 300). As stated above, data compression isperformed on a per data block basis. Accordingly, the first input datablock in the input data stream is input into the counter module 10 thatcounts the size of the data block (step 302). The data block is thenstored in the buffer 20 (step 304). The data block is then sent to theencoder module 30 and compressed by each (enabled) encoder E1 . . . En(step 306). Upon completion of the encoding of the input data block, anencoded data block is output from each (enabled) encoder E1 . . . En andmaintained in a corresponding buffer (step 308), and the encoded datablock size is counted (step 310).

[0066] Next, a compression ratio is calculated for each encoded datablock by taking the ratio of the size of the input data block (asdetermined by the input counter 10) to the size of each encoded datablock output from the enabled encoders (step 312). Each compressionratio is then compared with an a priori-specified compression ratiothreshold (step 314). It is to be understood that the threshold limitmay be specified as any value inclusive of data expansion, no datacompression or expansion, or any arbitrarily desired compression limit.It is to be further understood that notwithstanding that the currentlimit for lossless data compression is the entropy limit (the presentdefinition of information content) for the data, the present inventiondoes not preclude the use of future developments in lossless datacompression that may increase lossless data compression ratios beyondwhat is currently known within the art.

[0067] After the compression ratios are compared with the threshold, adetermination is made as to whether the compression ratio of at leastone of the encoded data blocks exceeds the threshold limit (step 316).If there are no encoded data blocks having a compression ratio thatexceeds the compression ratio threshold limit (negative determination instep 316), then the original unencoded input data block is selected foroutput and a null data compression type descriptor is appended thereto(step 318). A null data compression type descriptor is defined as anyrecognizable data token or descriptor that indicates no data encodinghas been applied to the input data block. Accordingly, the unencodedinput data block with its corresponding null data compression typedescriptor is then output for subsequent data processing, storage, ortransmittal (step 320).

[0068] On the other hand, if one or more of the encoded data blockspossess a compression ratio greater than the compression ratio thresholdlimit (affirmative result in step 316), then the encoded data blockhaving the greatest compression ratio is selected (step 322). Anappropriate data compression type descriptor is then appended (step324). A data compression type descriptor is defined as any recognizabledata token or descriptor that indicates which data encoding techniquehas been applied to the data. It is to be understood that, sinceencoders of the identical type may be applied in parallel to enhanceencoding speed (as discussed above), the data compression typedescriptor identifies the corresponding encoding technique applied tothe encoded data block, not necessarily the specific encoder. Theencoded data block having the greatest compression ratio along with itscorresponding data compression type descriptor is then output forsubsequent data processing, storage, or transmittal (step 326).

[0069] After the encoded data block or the unencoded data input datablock is output (steps 326 and 320), a determination is made as towhether the input data stream contains additional data blocks to beprocessed (step 328). If the input data stream includes additional datablocks (affirmative result in step 328), the next successive data blockis received (step 330), its block size is counted (return to step 302)and the data compression process in repeated. This process is iteratedfor each data block in the input data stream. Once the final input datablock is processed (negative result in step 328), data compression ofthe input data stream is finished (step 322).

[0070] Since a multitude of data types may be present within a giveninput data block, it is often difficult and/or impractical to predictthe level of compression that will be achieved by a specific encoder.Consequently, by processing the input data blocks with a plurality ofencoding techniques and comparing the compression results, content freedata compression is advantageously achieved. It is to be appreciatedthat this approach is scalable through future generations of processors,dedicated hardware, and software. As processing capacity increases andcosts reduce, the benefits provided by the present invention willcontinue to increase. It should again be noted that the presentinvention may employ any lossless data encoding technique.

[0071] Referring now to FIG. 4, a block diagram illustrates a contentindependent data compression system according to another embodiment ofthe present invention. The data compression system depicted in FIG. 4 issimilar to the data compression system of FIG. 2 except that theembodiment of FIG. 4 includes an enhanced metric functionality forselecting an optimal encoding technique. In particular, each of theencoders E1 . . . En in the encoder module 30 is tagged with acorresponding one of user-selected encoder desirability factors 70.Encoder desirability is defined as an a priori user specified factorthat takes into account any number of user considerations including, butnot limited to, compatibility of the encoded data with existingstandards, data error robustness, or any other aggregation of factorsthat the user wishes to consider for a particular application. Eachencoded data block output from the encoder module 30 has a correspondingdesirability factor appended thereto. A figure of merit module 80,operatively coupled to the compression ratio module 50 and thedescriptor module 60, is provided for calculating a figure of merit foreach of the encoded data blocks which possess a compression ratiogreater than the compression ratio threshold limit. The figure of meritfor each encoded data block is comprised of a weighted average of the apriori user specified threshold and the corresponding encoderdesirability factor. As discussed below in further detail with referenceto FIGS. 5a and 5 b, the figure of merit substitutes the a priori usercompression threshold limit for selecting and outputting encoded datablocks.

[0072] The operation of the data compression system of FIG. 4 will nowbe discussed in further detail with reference to the flow diagram ofFIGS. 5a and 5 b. A data stream comprising one or more data blocks isinput into the data compression system and the first data block in thestream is received (step 500). The size of the first data block is thendetermined by the counter module 10 (step 502). The data block is thenstored in the buffer 20 (step 504). The data block is then sent to theencoder module 30 and compressed by each (enabled) encoder in theencoder set E1 . . . En (step 506). Each encoded data block processed inthe encoder module 30 is tagged with an encoder desirability factor thatcorresponds the particular encoding technique applied to the encodeddata block (step 508). Upon completion of the encoding of the input datablock, an encoded data block with its corresponding desirability factoris output from each (enabled) encoder E1 . . . En and maintained in acorresponding buffer (step 510), and the encoded data block size iscounted (step 512).

[0073] Next, a compression ratio obtained by each enabled encoder iscalculated by taking the ratio of the size of the input data block (asdetermined by the input counter 10) to the size of the encoded datablock output from each enabled encoder (step 514). Each compressionratio is then compared with an a priori-specified compression ratiothreshold (step 516). A determination is made as to whether thecompression ratio of at least one of the encoded data blocks exceeds thethreshold limit (step 518). If there are no encoded data blocks having acompression ratio that exceeds the compression ratio threshold limit(negative determination in step 518), then the original unencoded inputdata block is selected for output and a null data compression typedescriptor (as discussed above) is appended thereto (step 520).Accordingly, the original unencoded input data block with itscorresponding null data compression type descriptor is then output forsubsequent data processing, storage, or transmittal (step 522).

[0074] On the other hand, if one or more of the encoded data blockspossess a compression ratio greater than the compression ratio thresholdlimit (affirmative result in step 518), then a figure of merit iscalculated for each encoded data block having a compression ratio whichexceeds the compression ratio threshold limit (step 524). Again, thefigure of merit for a given encoded data block is comprised of aweighted average of the a priori user specified threshold and thecorresponding encoder desirability factor associated with the encodeddata block. Next, the encoded data block having the greatest figure ofmerit is selected for output (step 526). An appropriate data compressiontype descriptor is then appended (step 528) to indicate the dataencoding technique applied to the encoded data block. The encoded datablock (which has the greatest figure of merit) along with itscorresponding data compression type descriptor is then output forsubsequent data processing, storage, or transmittal (step 530).

[0075] After the encoded data block or the unencoded input data block isoutput (steps 530 and 522), a determination is made as to whether theinput data stream contains additional data blocks to be processed (step532). If the input data stream includes additional data blocks(affirmative result in step 532), then the next successive data block isreceived (step 534), its block size is counted (return to step 502) andthe data compression process is iterated for each successive data blockin the input data stream. Once the final input data block is processed(negative result in step 532), data compression of the input data streamis finished (step 536).

[0076] Referring now to FIG. 6, a block diagram illustrates a datacompression system according to another embodiment of the presentinvention. The data compression system depicted in FIG. 6 is similar tothe data compression system discussed in detail above with reference toFIG. 2 except that the embodiment of FIG. 6 includes an a priorispecified timer that provides real-time or pseudo real-time output data.In particular, an interval timer 90, operatively coupled to the encodermodule 30, is preloaded with a user specified time value. The role ofthe interval timer (as will be explained in greater detail below withreference to FIGS. 7a and 7 b) is to limit the processing time for eachinput data block processed by the encoder module 30 so as to ensure thatthe real-time, pseudo real-time, or other time critical nature of thedata compression processes is preserved.

[0077] The operation of the data compression system of FIG. 6 will nowbe discussed in further detail with reference to the flow diagram ofFIGS. 7a and 7 b. A data stream comprising one or more data blocks isinput into the data compression system and the first data block in thedata stream is received (step 700), and its size is determined by thecounter module 10 (step 702). The data block is then stored in buffer 20(step 704).

[0078] Next, concurrent with the completion of the receipt and countingof the first data block, the interval timer 90 is initialized (step 706)and starts counting towards a user-specified time limit. The input datablock is then sent to the encoder module 30 wherein data compression ofthe data block by each (enabled) encoder E1 . . . En commences (step708). Next, a determination is made as to whether the user specifiedtime expires before the completion of the encoding process (steps 710and 712). If the encoding process is completed before or at theexpiration of the timer, i.e., each encoder (E1 through En) completesits respective encoding process (negative result in step 710 andaffirmative result in step 712), then an encoded data block is outputfrom each (enabled) encoder E1 . . . En and maintained in acorresponding buffer (step 714).

[0079] On the other hand, if the timer expires (affirmative result in710), the encoding process is halted (step 716). Then, encoded datablocks from only those enabled encoders E1 . . . En that have completedthe encoding process are selected and maintained in buffers (step 718).It is to be appreciated that it is not necessary (or in some casesdesirable) that some or all of the encoders complete the encodingprocess before the interval timer expires. Specifically, due to encoderdata dependency and natural variation, it is possible that certainencoders may not operate quickly enough and, therefore, do not complywith the timing constraints of the end use. Accordingly, the time limitensures that the real-time or pseudo real-time nature of the dataencoding is preserved.

[0080] After the encoded data blocks are buffered (step 714 or 718), thesize of each encoded data block is counted (step 720). Next, acompression ratio is calculated for each encoded data block by takingthe ratio of the size of the input data block (as determined by theinput counter 10) to the size of the encoded data block output from eachenabled encoder (step 722). Each compression ratio is then compared withan a priori-specified compression ratio threshold (step 724). Adetermination is made as to whether the compression ratio of at leastone of the encoded data blocks exceeds the threshold limit (step 726).If there are no encoded data blocks having a compression ratio thatexceeds the compression ratio threshold limit (negative determination instep 726), then the original unencoded input data block is selected foroutput and a null data compression type descriptor is appended thereto(step 728). The original unencoded input data block with itscorresponding null data compression type descriptor is then output forsubsequent data processing, storage, or transmittal (step 730).

[0081] On the other hand, if one or more of the encoded data blockspossess a compression ratio greater than the compression ratio thresholdlimit (affirmative result in step 726), then the encoded data blockhaving the greatest compression ratio is selected (step 732). Anappropriate data compression type descriptor is then appended (step734). The encoded data block having the greatest compression ratio alongwith its corresponding data compression type descriptor is then outputfor subsequent data processing, storage, or transmittal (step 736).

[0082] After the encoded data block or the unencoded input data block isoutput (steps 730 or 736), a determination is made as to whether theinput data stream contains additional data blocks to be processed (step738). If the input data stream includes additional data blocks(affirmative result in step 738), the next successive data block isreceived (step 740), its block size is counted (return to step 702) andthe data compression process in repeated. This process is iterated foreach data block in the input data stream, with each data block beingprocessed within the user-specified time limit as discussed above. Oncethe final input data block is processed (negative result in step 738),data compression of the input data stream is complete (step 742).

[0083] Referring now to FIG. 8, a block diagram illustrates a contentindependent data compression system according to another embodiment ofthe present system. The data compression system of FIG. 8 incorporatesall of the features discussed above in connection with the systemembodiments of FIGS. 2, 4, and 6. For example, the system of FIG. 8incorporates both the a priori specified timer for providing real-timeor pseudo real-time of output data, as well as the enhanced metric forselecting an optimal encoding technique. Based on the foregoingdiscussion, the operation of the system of FIG. 8 is understood by thoseskilled in the art.

[0084] Referring now to FIG. 9, a block diagram illustrates a datacompression system according to a preferred embodiment of the presentinvention. The system of FIG. 9 contains many of the features of theprevious embodiments discussed above. However, this embodimentadvantageously includes a cascaded encoder module 30 c having anencoding architecture comprising a plurality of sets ofserially-cascaded encoders Em,n, where “m” refers to the encoding path(i.e., the encoder set) and where “n” refers to the number of encodersin the respective path. It is to be understood that each set of seriallycascaded encoders can include any number of disparate and/or similarencoders (i.e., n can be any value for a given path m).

[0085] The system of FIG. 9 also includes a output buffer module 40 cwhich comprises a plurality of buffer/counters B/C m,n, each associatedwith a corresponding one of the encoders Em,n. In this embodiment, aninput data block is sequentially applied to successive encoders (encoderstages) in the encoder path so as to increase the data compressionratio. For example, the output data block from a first encoder E1,1, isbuffered and counted in B/C1,1, for subsequent processing by a secondencoder E1,2. Advantageously, these parallel sets of sequential encodersare applied to the input data stream to effect content free losslessdata compression. This embodiment provides for multi-stage sequentialencoding of data with the maximum number of encoding steps subject tothe available real-time, pseudo real-time, or other timing constraints.

[0086] As with each previously discussed embodiment, the encoders Em,nmay include those lossless encoding techniques currently well knownwithin the art, including: run length, Huffman, Lempel-Ziv DictionaryCompression, arithmetic coding, data compaction, and data nullsuppression. Encoding techniques are selected based upon their abilityto effectively encode different types of input data. A full complementof encoders provides for broad coverage of existing and future datatypes. The input data blocks may be applied simultaneously to theencoder paths (i.e., the encoder paths may operate in parallel,utilizing task multiplexing on a single central processor, or viadedicated hardware, or by executing on a plurality of processor ordedicated hardware systems, or any combination thereof). In addition, aninput data block may be sequentially applied to the encoder paths.Moreover, each serially cascaded encoder path may comprise a fixed(predetermined) sequence of encoders or a random sequence of encoders.Advantageously, by simultaneously or sequentially processing input datablocks via a plurality of sets of serially cascaded encoders, contentfree data compression is achieved.

[0087] The operation of the data compression system of FIG. 9 will nowbe discussed in further detail with reference to the flow diagram ofFIGS. 10a and 10 b. A data stream comprising one or more data blocks isinput into the data compression system and the first data block in thedata stream is received (step 100), and its size is determined by thecounter module 10 (step 102). The data block is then stored in buffer 20(step 104).

[0088] Next, concurrent with the completion of the receipt and countingof the first data block, the interval timer 90 is initialized (step 106)and starts counting towards a user-specified time limit. The input datablock is then sent to the cascade encoder module 30C wherein the inputdata block is applied to the first encoder (i.e., first encoding stage)in each of the cascaded encoder paths E1,1 . . . Em,1 (step 108). Next,a determination is made as to whether the user specified time expiresbefore the completion of the first stage encoding process (steps 110 and112). If the first stage encoding process is completed before theexpiration of the timer, i.e., each encoder (E1,1 . . . Em,1) completesits respective encoding process (negative result in step 110 andaffirmative result in step 112), then an encoded data block is outputfrom each encoder E1,1 . . . Em,1 and maintained in a correspondingbuffer (step 114). Then for each cascade encoder path, the output of thecompleted encoding stage is applied to the next successive encodingstage in the cascade path (step 116). This process (steps 110, 1 12,114, and 116) is repeated until the earlier of the timer expiration(affirmative result in step 110) or the completion of encoding by eachencoder stage in the serially cascaded paths, at which time the encodingprocess is halted (step 118).

[0089] Then, for each cascade encoder path, the buffered encoded datablock output by the last encoder stage that completes the encodingprocess before the expiration of the timer is selected for furtherprocessing (step 120). Advantageously, the interim stages of themulti-stage data encoding process are preserved. For example, theresults of encoder E1,1 are preserved even after encoder E1,2 beginsencoding the output of encoder E1,1. If the interval timer expires afterencoder E1,1 completes its respective encoding process but beforeencoder E1,2 completes its respective encoding-process, the encoded datablock from encoder E1,1 is complete and is utilized for calculating thecompression ratio for the corresponding encoder path. The incompleteencoded data block from encoder E1,2 is either discarded or ignored.

[0090] It is to be appreciated that it is not necessary (or in somecases desirable) that some or all of the encoders in the cascade encoderpaths complete the encoding process before the interval timer expires.Specifically, due to encoder data dependency, natural variation and thesequential application of the cascaded encoders, it is possible thatcertain encoders may not operate quickly enough and therefore do notcomply with the timing constraints of the end use. Accordingly, the timelimit ensures that the real-time or pseudo real-time nature of the dataencoding is preserved.

[0091] After the encoded data blocks are selected (step 120), the sizeof each encoded data block is counted (step 122). Next, a compressionratio is calculated for each encoded data block by taking the ratio ofthe size of the input data block (as determined by the input counter 10)to the size of the encoded data block output from each encoder (step124). Each compression ratio is then compared with an a priori-specifiedcompression ratio threshold (step 126). A determination is made as towhether the compression ratio of at least one of the encoded data blocksexceeds the threshold limit (step 128). If there are no encoded datablocks having a compression ratio that exceeds the compression ratiothreshold limit (negative determination in step 128), then the originalunencoded input data block is selected for output and a null datacompression type descriptor is appended thereto (step 130). The originalunencoded data block and its corresponding null data compression typedescriptor is then output for subsequent data processing, storage, ortransmittal (step 132).

[0092] On the other hand, if one or more of the encoded data blockspossess a compression ratio greater than the compression ratio thresholdlimit (affirmative result in step 128), then a figure of merit iscalculated for each encoded data block having a compression ratio whichexceeds the compression ratio threshold limit (step 134). Again, thefigure of merit for a given encoded data block is comprised of aweighted average of the a priori user specified threshold and thecorresponding encoder desirability factor associated with the encodeddata block. Next, the encoded data block having the greatest figure ofmerit is selected (step 136). An appropriate data compression typedescriptor is then appended (step 138) to indicate the data encodingtechnique applied to the encoded data block. For instance, the data typecompression descriptor can indicate that the encoded data block wasprocessed by either a single encoding type, a plurality of sequentialencoding types, and a plurality of random encoding types. The encodeddata block (which has the greatest figure of merit) along with itscorresponding data compression type descriptor is then output forsubsequent data processing, storage, or transmittal (step 140).

[0093] After the unencoded data block or the encoded data input datablock is output (steps 132 and 140), a determination is made as towhether the input data stream contains additional data blocks to beprocessed (step 142). If the input data stream includes additional datablocks (affirmative result in step 142), then the next successive datablock is received (step 144), its block size is counted (return to step102) and the data compression process is iterated for each successivedata block in the input data stream. Once the final input data block isprocessed (negative result in step 142), data compression of the inputdata stream is finished (step 146).

[0094] Referring now to FIG. 11, a block diagram illustrates a datadecompression system according to one embodiment of the presentinvention. The data decompression system preferably includes an inputbuffer 1100 that receives as input an uncompressed or compressed datastream comprising one or more data blocks. The data blocks may range insize from individual bits through complete files or collections ofmultiple files. Additionally, the data block size may be fixed orvariable. The input data buffer 1100 is preferably included (notrequired) to provide storage of input data for various hardwareimplementations. A descriptor extraction module 1102 receives thebuffered (or unbuffered) input data block and then parses, lexically,syntactically, or otherwise analyzes the input data block using methodsknown by those skilled in the art to extract the data compression typedescriptor associated with the data block. The data compression typedescriptor may possess values corresponding to null (no encodingapplied), a single applied encoding technique, or multiple encodingtechniques applied in a specific or random order (in accordance with thedata compression system embodiments and methods discussed above).

[0095] A decoder module 1104 includes a plurality of decoders D1 . . .Dn for decoding the input data block using a decoder, set of decoders,or a sequential set of decoders corresponding to the extractedcompression type descriptor. The decoders D1 . . . Dn may include thoselossless encoding techniques currently well known within the art,including: run length, Huffman, Lempel-Ziv Dictionary Compression,arithmetic coding, data compaction, and data null suppression. Decodingtechniques are selected based upon their ability to effectively decodethe various different types of encoded input data generated by the datacompression systems described above or originating from any otherdesired source. As with the data compression systems discussed above,the decoder module 1104 may include multiple decoders of the same typeapplied in parallel so as to reduce the data decoding time.

[0096] The data decompression system also includes an output data buffer1106 for buffering the decoded data block output from the decoder module1104.

[0097] The operation of the data decompression system of FIG. 11 will bediscussed in further detail with reference to the flow diagram of FIG.12. A data stream comprising one or more data blocks of compressed oruncompressed data is input into the data decompression system and thefirst data block in the stream is received (step 1200) and maintained inthe buffer (step 1202). As with the data compression systems discussedabove, data decompression is performed on a per data block basis. Thedata compression type descriptor is then extracted from the input datablock (step 1204). A determination is then made as to whether the datacompression type descriptor is null (step 1206). If the data compressiontype descriptor is determined to be null (affirmative result in step1206), then no decoding is applied to the input data block and theoriginal undecoded data block is output (or maintained in the outputbuffer) (step 1208).

[0098] On the other hand, if the data compression type descriptor isdetermined to be any value other than null (negative result in step1206), the corresponding decoder or decoders are then selected (step1210) from the available set of decoders D1 . . . Dn in the decodingmodule 1104. It is to be understood that the data compression typedescriptor may mandate the application of: a single specific decoder, anordered sequence of specific decoders, a random order of specificdecoders, a class or family of decoders, a mandatory or optionalapplication of parallel decoders, or any combination or permutationthereof. The input data block is then decoded using the selecteddecoders (step 1212), and output (or maintained in the output buffer1106) for subsequent data processing, storage, or transmittal (step1214). A determination is then made as to whether the input data streamcontains additional data blocks to be processed (step 1216). If theinput data stream includes additional data blocks (affirmative result instep 1216), the next successive data block is received (step 1220), andbuffered (return to step 1202). Thereafter, the data decompressionprocess is iterated for each data block in the input data stream. Oncethe final input data block is processed (negative result in step 1216),data decompression of the input data stream is finished (step 1218).

[0099] In other embodiments of the present invention described below,data compression is achieved using a combination of content dependentdata compression and content independent data compression. For example,FIGS. 13a and 13 b are block diagrams illustrating a data compressionsystem employing both content independent and content dependent datacompression according to one embodiment of the present invention,wherein content independent data compression is applied to a data blockwhen the content of the data block cannot be identified or is notassociable with a specific data compression algorithm. The datacompression system comprises a counter module 10 that receives as inputan uncompressed or compressed data stream. It is to be understood thatthe system processes the input data stream in data blocks that may rangein size from individual bits through complete files or collections ofmultiple files. Additionally, the data block size may be fixed orvariable. The counter module 10 counts the size of each input data block(i.e., the data block size is counted in bits, bytes, words, anyconvenient data multiple or metric, or any combination thereof).

[0100] An input data buffer 20, operatively connected to the countermodule 10, may be provided for buffering the input data stream in orderto output an uncompressed data stream in the event that, as discussed infurther detail below, every encoder fails to achieve a level ofcompression that exceeds a priori specified content independent orcontent dependent minimum compression ratio thresholds. It is to beunderstood that the input data buffer 20 is not required forimplementing the present invention.

[0101] A content dependent data recognition module 1300 analyzes theincoming data stream to recognize data types, data structures, datablock formats, file substructures, file types, and/or any otherparameters that may be indicative of either the data type/content of agiven data block or the appropriate data compression algorithm oralgorithms (in serial or in parallel) to be applied. Optionally, a datafile recognition list(s) or algorithm(s) 1310 module may be employed tohold and/or determine associations between recognized data parametersand appropriate algorithms. Each data block that is recognized by thecontent data compression module 1300 is routed to a content dependentencoder module 1320, if not the data is routed to the contentindependent encoder module 30.

[0102] A content dependent encoder module 1320 is operatively connectedto the content dependent data recognition module 1300 and comprises aset of encoders D1, D2, D3 . . . Dm. The encoder set D1, D2, D3 . . . Dmmay include any number “n” of those lossless or lossy encodingtechniques currently well known within the art such as MPEG4, variousvoice codecs, MPEG3, AC3, AAC, as well as lossless algorithms such asrun length, Huffman, Lempel-Ziv Dictionary Compression, arithmeticcoding, data compaction, and data null suppression. It is to beunderstood that the encoding techniques are selected based upon theirability to effectively encode different types of input data. It is to beappreciated that a full complement of encoders and or codecs arepreferably selected to provide a broad coverage of existing and futuredata types.

[0103] The content independent encoder module 30, which is operativelyconnected to the content dependent data recognition module 1300,comprises a set of encoders E1, E2, E3 . . . En. The encoder set E1, E2,E3 . . . En may include any number “n” of those lossless encodingtechniques currently well known within the art such as run length,Huffman, Lempel-Ziv Dictionary Compression, arithmetic coding, datacompaction, and data null suppression. Again, it is to be understoodthat the encoding techniques are selected based upon their ability toeffectively encode different types of input data. It is to beappreciated that a full complement of encoders are preferably selectedto provide a broad coverage of existing and future data types.

[0104] The encoder modules (content dependent 1320 and contentindependent 30) selectively receive the buffered input data blocks (orunbuffered input data blocks from the counter module 10) from module1300 based on the results of recognition. Data compression is performedby the respective encoder modules wherein some or all of the encoders D1. . . Dm or E1 . . . En processes a given input data block and outputs acorresponding set of encoded data blocks. It is to be appreciated thatthe system affords a user the option to enable/disable any one or moreof the encoders D1 . . . Dm and E1 . . . En prior to operation. As isunderstood by those skilled in the art, such feature allows the user totailor the operation of the data compression system for specificapplications. It is to be further appreciated that the encoding processmay be performed either in parallel or sequentially. In particular, theencoder set D1 through Dm of encoder module 1320 and/or the encoder setE1 through En of encoder module 30 may operate in parallel (i.e.,simultaneously processing a given input data block by utilizing taskmultiplexing on a single central processor, via dedicated hardware, byexecuting on a plurality of processor or dedicated hardware systems, orany combination thereof). In addition, encoders D1 through Dm and E1through En may operate sequentially on a given unbuffered or bufferedinput data block. This process is intended to eliminate the complexityand additional processing overhead associated with multiplexingconcurrent encoding techniques on a single central processor and/ordedicated hardware, set of central processors and/or dedicated hardware,or any achievable combination. It is to be further appreciated thatencoders of the identical type may be applied in parallel to enhanceencoding speed. For instance, encoder E1 may comprise two parallelHuffman encoders for parallel processing of an input data block. Itshould be further noted that one or more algorithms may be implementedin dedicated hardware such as an MPEG4 or MP3 encoding integratedcircuit.

[0105] Bluffer/counter modules 1330 and 40 are operatively connected totheir respective encoding modules 1320 and 30, for buffering andcounting the size of each of the encoded data blocks output from therespective encoder modules. Specifically, the content dependentbuffer/counter 1330 comprises a plurality of buffer/counters BCD1, BCD2,BCD3 . . . BCDm, each operatively associated with a corresponding one ofthe encoders D1 . . . Dm. Similarly the content independentbuffer/counters BCE1, BCE2, BCE3 . . . BCEn, each operatively associatedwith a corresponding one of the encoders E1 . . . En. A compressionratio module 1340, operatively connected to the content dependent outputbuffer/counters 1330 and content independent buffer/counters 40determines the compression ratio obtained for each of the enabledencoders D1 . . . Dm and or E1 . . . En by taking the ratio of the sizeof the input data block to the size of the output data block stored inthe corresponding buffer/counters BCD1, BCD2, BCD3 . . . BCDm and orBCE1, BCE2, BCE3 . . . BCEn. In addition, the compression ratio module1340 compares each compression ratio with an a priori-specifiedcompression ratio threshold limit to determine if at least one of theencoded data blocks output from the enabled encoders BCD1, BCD2, BCD3 .. . BCDm and or BCE1, BCE2, BCE3 . . . BCEn achieves a compression thatmeets an a priori-specified threshold. As is understood by those skilledin the art, the threshold limit may be specified as any value inclusiveof data expansion, no data compression or expansion, or any arbitrarilydesired compression limit. It should be noted that different thresholdvalues may be applied to content dependent and content independentencoded data. Further these thresholds may be adaptively modified basedupon enabled encoders in either or both the content dependent or contentindependent encoder sets, along with any associated parameters. Acompression type description module 1350, operatively coupled to thecompression ratio module 1340, appends a corresponding compression typedescriptor to each encoded data block which is selected for output so asto indicate the type of compression format of the encoded data block.

[0106] A mode of operation of the data compression system of FIGS. 13aand 13 b will now be discussed with reference to the flow diagrams ofFIGS. 14a-14 d, which illustrates a method for performing datacompression using a combination of content dependent and contentindependent data compression. In general, content independent datacompression is applied to a given data block when the content of a datablock cannot be identified or is not associated with a specific datacompression algorithm. More specifically, referring to FIG. 14a, a datastream comprising one or more data blocks is input into the datacompression system and the first data block in the stream is received(step 1400). As stated above, data compression is performed on a perdata block basis. As previously stated a data block may represent anyquantity of data from a single bit through a multiplicity of files orpackets and may vary from block to block. Accordingly, the first inputdata block in the input data stream is input into the counter module 10that counts the size of the data block (step 1402). The data block isthen stored in the buffer 20 (step 1404). The data block is thenanalyzed on a per block or multi-block basis by the content dependentdata recognition module 1300 (step 1406). If the data stream content isnot recognized utilizing the recognition list(s) or algorithms(s) module1310 (step 1408) the data is routed to the content independent encodermodule 30 and compressed by each (enabled) encoder E1 . . . En (step1410). Upon completion of the encoding of the input data block, anencoded data block is output from each (enabled) encoder E1 . . . En andmaintained in a corresponding buffer (step 1412), and the encoded datablock size is counted (step 1414).

[0107] Next, a compression ratio is calculated for each encoded datablock by taking the ratio of the size of the input data block (asdetermined by the input counter 10 to the size of each encoded datablock output from the enabled encoders (step 1416). Each compressionratio is then compared with an a priori-specified compression ratiothreshold (step 1418). It is to be understood that the threshold limitmay be specified as any value inclusive of data expansion, no datacompression or expansion, or any arbitrarily desired compression limit.It is to be further understood that notwithstanding that the currentlimit for lossless data compression is the entropy limit (the presentdefinition of information content) for the data, the present inventiondoes not preclude the use of future developments in lossless datacompression that may increase lossless data compression ratios beyondwhat is currently known within the art. Additionally the contentindependent data compression threshold may be different from the contentdependent threshold and either may be modified by the specific enabledencoders.

[0108] After the compression ratios are compared with the threshold, adetermination is made as to whether the compression ratio of at leastone of the encoded data blocks exceeds the threshold limit (step 1420).If there are no encoded data blocks having a compression ratio thatexceeds the compression ratio threshold limit (negative determination instep 1420), then the original unencoded input data block is selected foroutput and a null data compression type descriptor is appended thereto(step 1434). A null data compression type descriptor is defined as anyrecognizable data token or descriptor that indicates no data encodinghas been applied to the input data block. Accordingly, the unencodedinput data block with its corresponding null data compression typedescriptor is then output for subsequent data processing, storage, ortransmittal (step 1436).

[0109] On the other hand, if one or more of the encoded data blockspossess a compression ratio greater than the compression ratio thresholdlimit (affirmative result in step 1420), then the encoded data blockhaving the greatest compression ratio is selected (step 1422). Anappropriate data compression type descriptor is then appended (step1424). A data compression type descriptor is defined as any recognizabledata token or descriptor that indicates which data encoding techniquehas been applied to the data. It is to be understood that, sinceencoders of the identical type may be applied in parallel to enhanceencoding speed (as discussed above), the data compression typedescriptor identifies the corresponding encoding technique applied tothe encoded data block, not necessarily the specific encoder. Theencoded data block having the greatest compression ratio along with itscorresponding data compression type descriptor is then output forsubsequent data processing, storage, or transmittal (step 1426).

[0110] As previously stated the data block stored in the buffer 20 (step1404) is analyzed on a per block or multi-block basis by the contentdependent data recognition module 1300 (step 1406). If the data streamcontent is recognized utilizing the recognition list(s) or algorithms(s)module 1310 (step 1434) the appropriate content dependent algorithms areenabled and initialized (step 1436), and the data is routed to thecontent dependent encoder module 1320 and compressed by each (enabled)encoder D1 . . . Dm (step 1438). Upon completion of the encoding of theinput data block, an encoded data block is output from each (enabled)encoder D1 . . . Dm and maintained in a corresponding buffer (step1440), and the encoded data block size is counted (step 1442).

[0111] Next, a compression ratio is calculated for each encoded datablock by taking the ratio of the size of the input data block (asdetermined by the input counter 10 to the size of each encoded datablock output from the enabled encoders (step 1444). Each compressionratio is then compared with an a priori-specified compression ratiothreshold (step 1448). It is to be understood that the threshold limitmay be specified as any value inclusive of data expansion, no datacompression or expansion, or any arbitrarily desired compression limit.It is to be further understood that many of these algorithms may belossy, and as such the limits may be subject to or modified by an endtarget storage, listening, or viewing device. Further notwithstandingthat the current limit for lossless data compression is the entropylimit (the present definition of information content) for the data, thepresent invention does not preclude the use of future developments inlossless data compression that may increase lossless data compressionratios beyond what is currently known within the art. Additionally thecontent independent data compression threshold may be different from thecontent dependent threshold and either may be modified by the specificenabled encoders.

[0112] After the compression ratios are compared with the threshold, adetermination is made as to whether the compression ratio of at leastone of the encoded data blocks exceeds the threshold limit (step 1420).If there are no encoded data blocks having a compression ratio thatexceeds the compression ratio threshold limit (negative determination instep 1420), then the original unencoded input data block is selected foroutput and a null data compression type descriptor is appended thereto(step 1434). A null data compression type descriptor is defined as anyrecognizable data token or descriptor that indicates no data encodinghas been applied to the input data block. Accordingly, the unencodedinput data block with its corresponding null data compression typedescriptor is then output for subsequent data processing, storage, ortransmittal (step 1436).

[0113] On the other hand, if one or more of the encoded data blockspossess a compression ratio greater than the compression ratio thresholdlimit (affirmative result in step 1420), then the encoded data blockhaving the greatest compression ratio is selected (step 1422). Anappropriate data compression type descriptor is then appended (step1424). A data compression type descriptor is defined as any recognizabledata token or descriptor that indicates which data encoding techniquehas been applied to the data. It is to be understood that, sinceencoders of the identical type may be applied in parallel to enhanceencoding speed (as discussed above), the data compression typedescriptor identifies the corresponding encoding technique applied tothe encoded data block, not necessarily the specific encoder. Theencoded data block having the greatest compression ratio along with itscorresponding data compression type descriptor is then output forsubsequent data processing, storage, or transmittal (step 1426).

[0114] After the encoded data block or the unencoded data input datablock is output (steps 1426 and 1436), a determination is made as towhether the input data stream contains additional data blocks to beprocessed (step 1428). If the input data stream includes additional datablocks (affirmative result in step 1428), the next successive data blockis received (step 1432), its block size is counted (return to step 1402)and the data compression process in repeated. This process is iteratedfor each data block in the input data stream. Once the final input datablock is processed (negative result in step 1428), data compression ofthe input data stream is finished (step 1430).

[0115] Since a multitude of data types may be present within a giveninput data block, it is often difficult and/or impractical to predictthe level of compression that will be achieved by a specific encoder.Consequently, by processing the input data blocks with a plurality ofencoding techniques and comparing the compression results, content freedata compression is advantageously achieved. Further the encoding may belossy or lossless dependent upon the input data types. Further if thedata type is not recognized the default content independent losslesscompression is applied. It is not a requirement that this process bedeterministic—in fact a certain probability may be applied if occasionaldata loss is permitted. It is to be appreciated that this approach isscalable through future generations of processors, dedicated hardware,and software. As processing capacity increases and costs reduce, thebenefits provided by the present invention will continue to increase. Itshould again be noted that the present invention may employ any losslessdata encoding technique.

[0116]FIGS. 15a and 15 b are block diagrams illustrating a datacompression system employing both content independent and contentdependent data compression according to another embodiment of thepresent invention. The system in FIGS. 15a and 15 b is similar inoperation to the system of FIGS. 13a and 13 b in that contentindependent data compression is applied to a data block when the contentof the data block cannot be identified or is not associable with aspecific data compression algorithm. The system of FIGS. 15a and 15 badditionally performs content independent data compression on a datablock when the compression ratio obtained for the data block using thecontent dependent data compression does not meet a specified threshold.

[0117] A mode of operation of the data compression system of FIGS. 15aand 15 b will now be discussed with reference to the flow diagram ofFIGS. 16a-16 d, which illustrates a method for performing datacompression using a combination of content dependent and contentindependent data compression. A data stream comprising one or more datablocks is input into the data compression system and the first datablock in the stream is received (step 1600). As stated above, datacompression is performed on a per data block basis. As previously stateda data block may represent any quantity of data from a single bitthrough a multiplicity of files or packets and may vary from block toblock. Accordingly, the first input data block in the input data streamis input into the counter module 10 that counts the size of the datablock (step 1602). The data block is then stored in the buffer 20 (step1604). The data block is then analyzed on a per block or multi-blockbasis by the content dependent data recognition module 1300 (step 1606).If the data stream content is not recognized utilizing the recognitionlist(s) or algorithms(s) module 1310 (Step 1608) the data is routed tothe content independent encoder module 30 and compressed by each(enabled) encoder E1 . . . En (step 1610). Upon completion of theencoding of the input data block, an encoded data block is output fromeach (enabled) encoder E1 . . . En and maintained in a correspondingbuffer (step 1612), and the encoded data block size is counted (step1614).

[0118] Next, a compression ratio is calculated for each encoded datablock by taking the ratio of the size of the input data block (asdetermined by the input counter 10 to the size of each encoded datablock output from the enabled encoders (step 1616). Each compressionratio is then compared with an a priori-specified compression ratiothreshold (step 1618). It is to be understood that the threshold limitmay be specified as any value inclusive of data expansion, no datacompression or expansion, or any arbitrarily desired compression limit.It is to be further understood that notwithstanding that the currentlimit for lossless data compression is the entropy limit (the presentdefinition of information content) for the data, the present inventiondoes not preclude the use of future developments in lossless datacompression that may increase lossless data compression ratios beyondwhat is currently known within the art. Additionally the contentindependent data compression threshold may be different from the contentdependent threshold and either may be modified by the specific enabledencoders.

[0119] After the compression ratios are compared with the threshold, adetermination is made as to whether the compression ratio of at leastone of the encoded data blocks exceeds the threshold limit (step 1620).If there are no encoded data blocks having a compression ratio thatexceeds the compression ratio threshold limit (negative determination instep 1620), then the original unencoded input data block is selected foroutput and a null data compression type descriptor is appended thereto(step 1634). A null data compression type descriptor is defined as anyrecognizable data token or descriptor that indicates no data encodinghas been applied to the input data block. Accordingly, the unencodedinput data block with its corresponding null data compression typedescriptor is then output for subsequent data processing, storage, ortransmittal (step 1636).

[0120] On the other hand, if one or more of the encoded data blockspossess a compression ratio greater than the compression ratio thresholdlimit (affirmative result in step 1620), then the encoded data blockhaving the greatest compression ratio is selected (step 1622). Anappropriate data compression type descriptor is then appended (step1624). A data compression type descriptor is defined as any recognizabledata token or descriptor that indicates which data encoding techniquehas been applied to the data. It is to be understood that, sinceencoders of the identical type may be applied in parallel to enhanceencoding speed (as discussed above), the data compression typedescriptor identifies the corresponding encoding technique applied tothe encoded data block, not necessarily the specific encoder. Theencoded data block having the greatest compression ratio along with itscorresponding data compression type descriptor is then output forsubsequent data processing, storage, or transmittal (step 1626).

[0121] As previously stated the data block stored in the buffer 20 (step1604) is analyzed on a per block or multi-block basis by the contentdependent data recognition module 1300 (step 1606). If the data streamcontent is recognized utilizing the recognition list(s) or algorithms(s)module 1310 (step 1634) the appropriate content dependent algorithms areenabled and initialized (step 1636) and the data is routed to thecontent dependent encoder module 1620 and compressed by each (enabled)encoder D1 . . . Dm (step 1638). Upon completion of the encoding of theinput data block, an encoded data block is output from each (enabled)encoder D1 . . . Dm and maintained in a corresponding buffer (step1640), and the encoded data block size is counted (step 1642).

[0122] Next, a compression ratio is calculated for each encoded datablock by taking the ratio of the size of the input data block (asdetermined by the input counter 10 to the size of each encoded datablock output from the enabled encoders (step 1644). Each compressionratio is then compared with an a priori-specified compression ratiothreshold (step 1648). It is to be understood that the threshold limitmay be specified as any value inclusive of data expansion, no datacompression or expansion, or any arbitrarily desired compression limit.It is to be further understood that many of these algorithms may belossy, and as such the limits may be subject to or modified by an endtarget storage, listening, or viewing device. Further notwithstandingthat the current limit for lossless data compression is the entropylimit (the present definition of information content) for the data, thepresent invention does not preclude the use of future developments inlossless data compression that may increase lossless data compressionratios beyond what is currently known within the art. Additionally thecontent independent data compression threshold may be different from thecontent dependent threshold and either may be modified by the specificenabled encoders.

[0123] After the compression ratios are compared with the threshold, adetermination is made as to whether the compression ratio of at leastone of the encoded data blocks exceeds the threshold limit (step 1648).If there are no encoded data blocks having a compression ratio thatexceeds the compression ratio threshold limit (negative determination instep 1620), then the original unencoded input data block is routed tothe content independent encoder module 30 and the process resumes withcompression utilizing content independent encoders (step 1610).

[0124] After the encoded data block or the unencoded data input datablock is output (steps 1626 and 1636), a determination is made as towhether the input data stream contains additional data blocks to beprocessed (step 1628). If the input data stream includes additional datablocks (affirmative result in step 1628), the next successive data blockis received (step 1632), its block size is counted (return to step 1602)and the data compression process in repeated. This process is iteratedfor each data block in the input data stream. Once the final input datablock is processed (negative result in step 1628), data compression ofthe input data stream is finished (step 1630).

[0125]FIGS. 17a and 17 b are block diagrams illustrating a datacompression system employing both content independent and contentdependent data compression according to another embodiment of thepresent invention. The system in FIGS. 17a and 17 b is similar inoperation to the system of FIGS. 13a and 13 b in that contentindependent data compression is applied to a data block when the contentof the data block cannot be identified or is not associable with aspecific data compression algorithm. The system of FIGS. 17a and 17 badditionally uses a priori estimation algorithms or look-up tables toestimate the desirability of using content independent data compressionencoders and/or content dependent data compression encoders andselecting appropriate algorithms or subsets thereof based on suchestimation.

[0126] More specifically, a content dependent data recognition and orestimation module 1700 is utilized to analyze the incoming data streamfor recognition of data types, data structures, data block formats, filesubstructures, file types, or any other parameters that may beindicative of the appropriate data compression algorithm or algorithms(in serial or in parallel) to be applied. Optionally, a data filerecognition list(s) or algorithm(s) 1710 module may be employed to holdassociations between recognized data parameters and appropriatealgorithms. If the content data compression module recognizes a portionof the data, that portion is routed to the content dependent encodermodule 1320, if not the data is routed to the content independentencoder module 30. It is to be appreciated that process of recognition(modules 1700 and 1710) is not limited to a deterministic recognition,but may further comprise a probabilistic estimation of which encoders toselect for compression from the set of encoders of the content dependentmodule 1320 or the content independent module 30. For example, a methodmay be employed to compute statistics of a data block whereby adetermination that the locality of repetition of characters in a datastream is determined is high can suggest a text document, which may bebeneficially compressed with a lossless dictionary type algorithm.Further the statistics of repeated characters and relative frequenciesmay suggest a specific type of dictionary algorithm. Long strings willrequire a wide dictionary file while a wide diversity of strings maysuggest a deep dictionary. Statistics may also be utilized in algorithmssuch as Huffman where various character statistics will dictate thechoice of different Huffman compression tables. This technique is notlimited to lossless algorithms but may be widely employed with lossyalgorithms. Header information in frames for video files can imply aspecific data resolution. The estimator then may select the appropriatelossy compression algorithm and compression parameters (amount ofresolution desired). As shown in previous embodiments of the presentinvention, desirability of various algorithms and now associatedresolutions with lossy type algorithms may also be applied in theestimation selection process.

[0127] A mode of operation of the data compression system of FIGS. 17aand 17 b will now be discussed with reference to the flow diagrams ofFIGS. 18a-18 d. The method of FIGS. 18a-18 d use a priori estimationalgorithms or look-up tables to estimate the desirability or probabilityof using content independent data compression encoders or contentdependent data compression encoders, and select appropriate or desirablealgorithms or subsets thereof based on such estimates. A data streamcomprising one or more data blocks is input into the data compressionsystem and the first data block in the stream is received (step 1800).As stated above, data compression is performed on a per data blockbasis. As previously stated a data block may represent any quantity ofdata from a single bit through a multiplicity of files or packets andmay vary from block to block. Accordingly, the first input data block inthe input data stream is input into the counter module 10 that countsthe size of the data block (step 1802). The data block is then stored inthe buffer 20 (step 1804). The data block is then analyzed on a perblock or multi-block basis by the content dependent/content independentdata recognition module 1700 (step 1806). If the data stream content isnot recognized utilizing the recognition list(s) or algorithms(s) module1710 (step 1808) the data is to the content independent encoder module30. An estimate of the best content independent encoders is performed(step 1850) and the appropriate encoders are enabled and initialized asapplicable. The data is then compressed by each (enabled) encoder E1 . .. En (step 1810). Upon completion of the encoding of the input datablock, an encoded data block is output from each (enabled) encoder E1 .. . En and maintained in a corresponding buffer (step 1812), and theencoded data block size is counted (step 1814).

[0128] Next, a compression ratio is calculated for each encoded datablock by taking the ratio of the size of the input data block (asdetermined by the input counter 10 to the size of each encoded datablock output from the enabled encoders (step 1816). Each compressionratio is then compared with an a priori-specified compression ratiothreshold (step 1818). It is to be understood that the threshold limitmay be specified as any value inclusive of data expansion, no datacompression or expansion, or any arbitrarily desired compression limit.It is to be further understood that notwithstanding that the currentlimit for lossless data compression is the entropy limit (the presentdefinition of information content) for the data, the present inventiondoes not preclude the use of future developments in lossless datacompression that may increase lossless data compression ratios beyondwhat is currently known within the art. Additionally the contentindependent data compression threshold may be different from the contentdependent threshold and either may be modified by the specific enabledencoders.

[0129] After the compression ratios are compared with the threshold, adetermination is made as to whether the compression ratio of at leastone of the encoded data blocks exceeds the threshold limit (step 1820).If there are no encoded data blocks having a compression ratio thatexceeds the compression ratio threshold limit (negative determination instep 1820), then the original unencoded input data block is selected foroutput and a null data compression type descriptor is appended thereto(step 1834). A null data compression type descriptor is defined as anyrecognizable data token or descriptor that indicates no data encodinghas been applied to the input data block. Accordingly, the unencodedinput data block with its corresponding null data compression typedescriptor is then output for subsequent data processing, storage, ortransmittal (step 1836).

[0130] On the other hand, if one or more of the encoded data blockspossess a compression ratio greater than the compression ratio thresholdlimit (affirmative result in step 1820), then the encoded data blockhaving the greatest compression ratio is selected (step 1822). Anappropriate data compression type descriptor is then appended (step1824). A data compression type descriptor is defined as any recognizabledata token or descriptor that indicates which data encoding techniquehas been applied to the data. It is to be understood that, sinceencoders of the identical type may be applied in parallel to enhanceencoding speed (as discussed above), the data compression typedescriptor identifies the corresponding encoding technique applied tothe encoded data block, not necessarily the specific encoder. Theencoded data block having the greatest compression ratio along with itscorresponding data compression type descriptor is then output forsubsequent data processing, storage, or transmittal (step 1826).

[0131] As previously stated the data block stored in the buffer 20 (step1804) is analyzed on a per block or multi-block basis by the contentdependent data recognition module 1300 (step 1806). If the data streamcontent is recognized or estimated utilizing the recognition list(s) oralgorithms(s) module 1710 (affirmative result in step 1808) therecognized data type/file or block is selected based on a list oralgorithm (step 1838) and an estimate of the desirability of using theassociated content dependent algorithms can be determined (step 1840).For instance, even though a recognized data type may be associated withthree different encoders, an estimation of the desirability of usingeach encoder may result in only one or two of the encoders beingactually selected for use. The data is routed to the content dependentencoder module 1320 and compressed by each (enabled) encoder D1 . . . Dm(step 1842). Upon completion of the encoding of the input data block, anencoded data block is output from each (enabled) encoder D1 . . . Dm andmaintained in a corresponding buffer (step 1844), and the encoded datablock size is counted (step 1846).

[0132] Next, a compression ratio is calculated for each encoded datablock by taking the ratio of the size of the input data block (asdetermined by the input counter 10 to the size of each encoded datablock output from the enabled encoders (step 1848). Each compressionratio is then compared with an a priori-specified compression ratiothreshold (step 1850). It is to be understood that the threshold limitmay be specified as any value inclusive of data expansion, no datacompression or expansion, or any arbitrarily desired compression limit.It is to be further understood that many of these algorithms may belossy, and as such the limits may be subject to or modified by an endtarget storage, listening, or viewing device. Further notwithstandingthat the current limit for lossless data compression is the entropylimit (the present definition of information content) for the data, thepresent invention does not preclude the use of future developments inlossless data compression that may increase lossless data compressionratios beyond what is currently known within the art. Additionally thecontent independent data compression threshold may be different from thecontent dependent threshold and either may be modified by the specificenabled encoders.

[0133] After the compression ratios are compared with the threshold, adetermination is made as to whether the compression ratio of at leastone of the encoded data blocks exceeds the threshold limit (step 1820).If there are no encoded data blocks having a compression ratio thatexceeds the compression ratio threshold limit (negative determination instep 1820), then the original unencoded input data block is selected foroutput and a null data compression type descriptor is appended thereto(step 1834). A null data compression type descriptor is defined as anyrecognizable data token or descriptor that indicates no data encodinghas been applied to the input data block. Accordingly, the unencodedinput data block with its corresponding null data compression typedescriptor is then output for subsequent data processing, storage, ortransmittal (step 1836).

[0134] On the other hand, if one or more of the encoded data blockspossess a compression ratio greater than the compression ratio thresholdlimit (affirmative result in step 1820), then the encoded data blockhaving the greatest compression ratio is selected (step 1822). Anappropriate data compression type descriptor is then appended (step1824). A data compression type descriptor is defined as any recognizabledata token or descriptor that indicates which data encoding techniquehas been applied to the data. It is to be understood that, sinceencoders of the identical type may be applied in parallel to enhanceencoding speed (as discussed above), the data compression typedescriptor identifies the corresponding encoding technique applied tothe encoded data block, not necessarily the specific encoder. Theencoded data block having the greatest compression ratio along with itscorresponding data compression type descriptor is then output forsubsequent data processing, storage, or transmittal (step 1826).

[0135] After the encoded data block or the unencoded data input datablock is output (steps 1826 and 1836), a determination is made as towhether the input data stream contains additional data blocks to beprocessed (step 1828). If the input data stream includes additional datablocks (affirmative result in step 1428), the next successive data blockis received (step 1832), its block size is counted (return to step 1802)and the data compression process in repeated. This process is iteratedfor each data block in the input data stream. Once the final input datablock is processed (negative result in step 1828), data compression ofthe input data stream is finished (step 1830).

[0136] It is to be appreciated that in the embodiments described abovewith reference to FIGS. 13-18, an a priori specified time limit or anyother real-time requirement may be employed to achieve practical andefficient real-time operation.

[0137] Although illustrative embodiments have been described herein withreference to the accompanying drawings, it is to be understood that thepresent invention is not limited to those precise embodiments, and thatvarious other changes and modifications may be affected therein by oneskilled in the art without departing from the scope or spirit of theinvention. All such changes and modifications are intended to beincluded within the scope of the invention as defined by the appendedclaims.

What is claimed is:
 1. A method for compressing data, comprising thesteps of: analyzing a data block of an input data stream to identify adata type of the data block, the input data stream comprising aplurality of disparate data types; performing content dependent datacompression on the data block, if the data type of the data block isidentified; performing content independent data compression on the datablock, if the data type of the data block is not identified.
 2. Themethod of claim 1, wherein the step of performing content independentdata compression, comprises: encoding the data block with a plurality ofencoders to provide a plurality of encoded data blocks; determining acompression ratio obtained for each of the encoders; comparing each ofthe determined compression ratios with a first compression threshold;selecting for output the input data block and appending a nullcompression descriptor to the input data block, if all of the encodercompression ratios do not meet the first compression threshold; andselecting for output the encoded data block having the highestcompression ratio and appending a corresponding compression typedescriptor to the selected encoded data block, if at least one of thecompression ratios meet the first compression threshold.
 3. The methodof claim 1, wherein the step of performing content dependent compressioncomprises the steps of: selecting one or more encoders associated withthe identified data type and encoding the data block with the selectedencoders to provide a plurality of encoded data blocks; determining acompression ratio obtained for each of the selected encoders; comparingeach of the determined compression ratios with a second compressionthreshold; selecting for output the input data block and appending anull compression descriptor to the input data block, if all of theencoder compression do not meet the second compression threshold; andselecting for output the encoded data block having the highestcompression ratio and appending a corresponding compression typedescriptor to the selected encoded data block, if at least one of thecompression ratios meet the second compression threshold.
 4. The methodof claim 1, wherein the step of performing content independent datacompression on the data block, if the data type of the data block is notidentified, comprises the steps of: estimating a desirability of usingof one or more encoder types based one characteristics of the datablock; and compressing the data block using one or more desirableencoders.
 5. The method of claim 1, wherein the methods steps aretangibly embodied as program instructions on a program storage device,wherein the instructions are executable by a machine to perform themethod steps.
 6. The method of claim 1, wherein the step of performingcontent dependent data compression on the data block, if the data typeof the data block is identified, comprises the steps of: estimating adesirability of using of one or more encoder types based oncharacteristics of the data block; and compressing the data block usingone or more desirable encoders.
 7. The method of claim 1, wherein thestep of analyzing the data block comprises one of analyzing the datablock to recognize one of a data type, data structure, data blockformat, file substructure, file types and a combination thereof.
 8. Themethod of claim 7, further comprising the step of maintaining anassociation between encoder types and one of data types, datastructures, data block formats, file substructure, file types and acombination thereof.
 9. A method for compressing data, comprising thesteps of: analyzing a data block of an input data stream to identify adata type of the data block, the input data stream comprising aplurality of disparate data types; performing content dependent datacompression on the data block, if the data type of the data block isidentified; determining a compression ratio of the compressed data blockobtained using the content dependent compression and comparing thecompression ratio with a first compression threshold; and performingcontent independent data compression on the data block, if the data typeof the data block is not identified or if the compression ratio of thecompressed data block obtained using the content dependent compressiondoes not meet the first compression threshold.
 10. The method of claim9, wherein the step of performing content independent data compression,comprises: encoding the data block with a plurality of encoders toprovide a plurality of encoded data blocks; determining a compressionratio obtained for each of the encoders; comparing each of thedetermined compression ratios with a second compression threshold;selecting for output the input data block and appending a nullcompression descriptor to the input data block, if all of the encodercompression ratios do not meet the second compression threshold; andselecting for output the encoded data block having the highestcompression ratio and appending a corresponding compression typedescriptor to the selected encoded data block, if at least one of thecompression ratios meet the second compression threshold.
 11. The methodof claim 9, wherein the step of performing content dependent compressioncomprises the steps of: selecting one or more encoders associated withthe identified data type and encoding the data block with the selectedencoders to provide a plurality of encoded data blocks; determining acompression ratio obtained for each of the selected encoders; comparingeach of the determined compression ratios with the first compressionthreshold; selecting for output the input data block and appending anull compression descriptor to the input data block, if all of theencoder compression do not meet the first compression threshold; andselecting for output the encoded data block having the highestcompression ratio and appending a corresponding compression typedescriptor to the selected encoded data block, if at least one of thecompression ratios meet the first compression threshold.
 12. The methodof claim 9, wherein the step of performing content dependent datacompression when the data type of the data block is identified, furthercomprises the steps of: estimating a desirability of using of one ormore encoder types based one characteristics of the data block; andcompressing the data block using one or more desirable encoders.
 13. Themethod of claim 9, wherein the step of performing content independentdata compression on the data block, if the data type of the data blockis not identified, comprises the steps of: estimating a desirability ofusing of one or more encoder types based one characteristics of the datablock; and compressing the data block using one or more desirableencoders.
 14. The method of claim 9, wherein the methods steps aretangibly embodied as program instructions on a program storage device,wherein the instructions are executable by a machine to perform themethod steps.
 15. The method of claim 9, wherein the step of analyzingthe data block comprises one of analyzing the data block to recognizeone of a data type, data structure, data block format, filesubstructure, file types and a combination thereof.
 16. The method ofclaim 15, further comprising the step of maintaining an associationbetween encoder types and one of data types, data structures, data blockformats, file substructure, file types and a combination thereof.
 17. Aprogram storage device readable by a machine, tangibly embodying aprogram instructions executable by the machine to perform method stepsfor compressing data, the method steps comprising: receiving an inputdata stream comprising a plurality of disparate data types; compressingthe input data stream using each of a plurality of different encoders;and generating an encoded data stream by selectively combiningcompressed data blocks output from each of the encoders based oncompression ratios obtained by the encoders.
 18. The program storagedevice of claim 17, wherein the instructions for performing the step ofgenerating the encoded data stream comprise instructions for taggingeach compressed data block with a compression type descriptor.
 19. Theprogram storage device of claim 17, wherein the step of generating theencoded data stream comprises combining uncompressed data blocks fromthe input data stream with the compressed data blocks and tagging eachuncompressed data block with a null compression descriptor.
 20. Theprogram storage device of claim 17, wherein the instructions forperforming the step of compressing the input data stream compriseinstructions for compressing each data block in the input data streamusing each of the encoders, and wherein the instructions for performingthe step of generating the encoded data stream comprise instructionsfor: for each data block in the input stream, determining a compressionratio obtained from each of the encoders; selecting for output the inputdata block and appending a null compression descriptor to input the datablock, if no compression ratio exceeds a predetermined threshold; andselecting for output the encoded data block having the greatestcompression ratio associated therewith that meets the predeterminedthreshold and appending a compression type descriptor to the selectedencoded data block.
 21. The program storage device of claim 20, furthercomprising instructions for performing the step of applying apredetermined timing constraint to the compression process to providereal-time data compression of the input data stream.
 22. The programstorage device of 21, wherein the instructions for performing the stepof applying a predetermined time constraint comprise instructions forperforming the steps of: initializing a timer with a user-specified timeinterval upon commencing compression of an input data block; andterminating the encoding step upon the earlier of one of the expirationof the timer and the completion of the encoding of the input data blockby all of the plurality of encoders; wherein the step of determining thecompression ratios is only performed for the encoders that havecompleted encoding of the input data block before expiration of thetimer.